The Effect of Different Dimensionality Reduction Techniques on Machine Learning Overfitting Problem

In most conditions, it is a problematic mission for machine-learning model with data record, which has various attributes, to be trained. There always proportional relationship between the increase of features and arrival overfitting susceptible model. That observation occurred since not all characteristics are important. For example, some could only cause noisier. Dimensionality reduction techniques used overcome this matter. This paper presents detailed comparative study nine dimensionality methods. These methods missing-values ratio, low variance filter, high-correlation random forest, principal component analysis, linear discriminant backward feature elimination, forward construction, rough set theory. The effects on both training testing performance were compared two different datasets applied three models. models are, Artificial Neural Network (ANN), Support Vector Machine (SVM) Random Forest classifier (RFC). results proved that RFC was able achieve via limiting crisis. introduced showed general progress in accuracy efficiency against approaches. revealed minimize process while holding so near or better than original one.